Robot apparatus and method and system for controlling the action of the robot apparatus

ABSTRACT

A main robot apparatus generates a sound scale command at a command generating state (ST 2 ) to enter into a state of waiting for a reaction of a slave robot apparatus (ST 3 ). When the slave robot apparatus outputs a emotion expressing sound responsive to a sound scale command issued by the main robot apparatus, the main robot apparatus recognizes this emotion expressing sound to output the same emotion expressing sound. In a state of the reaction action (ST 4 ), the main robot apparatus selects an action (NumResponse), depending on the value of the variable NumResponse which has counted the number of times of the reactions to output the action.

TECHNICAL FIELD

[0001] This invention relates to a robot apparatus, an actioncontrolling method for controlling the action of the robot apparatus andan action controlling system for controlling the action of plural robotapparatus. More specifically, the present invention relates to a robotapparatus convenient for performing an autonomous action, an actioncontrolling method and an action controlling system for such robotapparatus

BACKGROUND ART

[0002] Recently, a robot apparatus, formed after the shape of an animal,such as a dog or a cat, has been presented to the market. Among theserobot apparatus, there is such an apparatus which acts autonomouslyresponsive to the information from outside or to an internal state ofthe robot apparatus. For example, the robot apparatus is designed to actunder a command from a user, as the information from outside, ordepending on the level of the emotion, indicating its internal state.

[0003] The following are the functions of the robot apparatus operatingresponsive to the level of the emotion indicating the internal state ofthe robot apparatus.

[0004] The robot apparatus has variable states A to D, as shown forexample in FIG. 1, and is designed to make transitions between thesestates under a preset condition or at preset timings. The states A to Dare defined by the postures assumed or actions performed by theapparatus when in the states concerned.

[0005] For example, when the robot apparatus is ‘angry’ when in thestate A, it correspondingly utters, e.g., ‘pi-ro-li-ro-li’ and, when therobot apparatus is ‘happy’ when in the state B, it correspondinglyutters ‘pi-po-pa’.

[0006] Moreover, by the functions of the robot apparatus, operatingunder a command from a user, the interaction between the robot apparatusand the user takes place. These functions may be enumerated by, forexample, those reacting to contact by the user from outside, and thosein action under a command from outside.

[0007] For example, by providing the robot apparatus with a touch sensordetecting the contact thereof with outside, and by causing the action ofthe robot apparatus responsive to the contact thereof with the touchsensor, there is realized the function of the robot apparatus reactingto contact from outside, such as by the user. The touch sensor may, forexample, be mounted on, e.g., a head of the robot apparatus.

[0008] As for the function of the robot apparatus, operating under anexternal command, the following technique may be used.

[0009] For example, an equipment outputting a scale command, as anexternal command, may be exploited. The scale command forms a commandfor causing the robot apparatus to express a preset action by the scalesound. The robot apparatus recognizes the scale command, output by theexternal control equipment, and performs the action corresponding to thescale command. The external communication equipment, outputting thisscale command, may be enumerated by, for example, a sound commander.Meanwhile, the technique of the control system of the apparatus,exploiting the scale command, has recently come to be established.

[0010] By such contact as the information from outside, or the scalecommand, the robot apparatus in the standby state detects ‘being patted’by soft touch at a touch sensor at the scalp or ‘being hit’ by strongtouch at the touch sensor as shown in FIG. 2. As a reaction thereto, therobot apparatus utters ‘pi-po-pa’ as the sound indicating the anger, ondetection of ‘being patted’, while uttering ‘pi-ro-li-ro-li’ as thesound indicating the sadness, on detection of ‘being hit’. If the scalecommand, for example, the sound scale language, is used, the robotapparatus utters, e.g., ‘pi-ro-li’ as an action corresponding torecognition of the scale command.

[0011] By the stimuli from outside by contact, or by the sound scalecommand, the action of the robot apparatus is affected, such as toenable the interaction between the user and the robot apparatus.

[0012] Meanwhile, the actions of the conventional robot apparatus aremerely determined by its own emotion or by the information afforded bythe user. If plural robot apparatus can interact with one another by wayof communication, and if the plural robot apparatus can react to oneanother and operate accordingly, the entertainment character of therobot apparatus will be improved further.

[0013] Moreover, if it is desirable for the plural robot apparatus tohave a dialog with one another, it may be an occurrence that the pluralrobot apparatus are not of the same machine type. For example, one ofthe robot apparatus may be apparatus of an original model, while anothermay be the product of a generic model, or the plural robot apparatus maybe furnished by different manufacturers. In such case, it is difficultfor the robot apparatus to react to one another.

[0014] The robot apparatus of the generic model may be afforded with thefunctions of the robot apparatus of the original model. In other words,there are occasions where the robot apparatus of the original model arenot afforded with the function owned by the robot apparatus of thegeneric model. Thus, there are occasions where the interaction functionis limited to the robot apparatus of the same machine type.

[0015] However, the entertainment character of the robot apparatus canbe enhanced by enabling the interaction between plural robot apparatushaving different interaction functions. Additionally, it may be saidthat such interaction among various robot apparatus lends itself toeffective utilization of the robot apparatus as resources.

DISCLOSURE OF THE INVENTION

[0016] In view of the above-described status of the art, it is an objectof the present invention to provide a robot apparatus that can have adialog with other robot apparatus, a method for controlling the actionof the robot apparatus and a system for controlling the robot apparatus.

[0017] For accomplishing the above object, the robot apparatus accordingto the present invention includes an action part, action detection meansfor detecting the information contained in an action output by anotherrobot apparatus, action part control means for controlling the actionpart, based on the information detected by the action detection means,and means for measuring the number of times of reactions of the otherrobot apparatus corresponding to the actions output by the action part.The action part control means outputs an action by the action partdepending on the number of times of reactions of the other robotapparatus.

[0018] The robot apparatus, constructed as described above, detects theinformation contained in the action output by other robot apparatus, byaction detection means, and controls the action part by action partcontrolling means to output an action. At this time, the action partcontrolling means controls the action part responsive to the number oftimes of reactions of the other robot apparatus as measured by the meansfor measuring the number of times of reactions. This permits the robotapparatus to act responsive to the action of the other robot apparatus.

[0019] For accomplishing the above object, the robot apparatus accordingto the present invention also includes action detection means fordetecting the information contained in an action output by another robotapparatus and action outputting means for outputting an actioncorresponding to the information as detected by the action detectionmeans.

[0020] The robot apparatus, constructed as described above, detects theinformation contained in the action output by the other robot apparatusto output the action corresponding to the information detected by theaction detection means to output the action corresponding to theinformation as detected by the action detection means by the actionoutputting means. This permits the robot apparatus to act responsive tothe action of the other robot apparatus.

[0021] For accomplishing the above object, an action controlling methodfor the robot apparatus including an action part according to thepresent invention includes an action detection step of detecting theinformation contained in an action output by another robot apparatus, anaction part controlling step of controlling the action part, based onthe information detected by the action detections step, and a step ofmeasuring the number of times of reactions of the other robot apparatusbased on the actions output by the action part. The action partcontrolling step outputs an action by the action part depending on thenumber of times of reactions of the other robot apparatus. This permitsthe robot apparatus to act responsive to the action of the other robotapparatus.

[0022] For accomplishing the above object, the action controlling methodfor the robot apparatus according to the present invention includes anaction detection step of detecting the information contained in anaction output by another robot apparatus and an action outputting stepof causing one of the robot apparatus to output an action correspondingto the information as detected by the action detection step.

[0023] For accomplishing the above object, an action controlling systemfor the robot apparatus according to the present invention includes aplurality of robot apparatus including action detection means fordetecting the information contained in an action output by thecounterpart robot apparatus, and action outputting means for outputtingan action corresponding to the information detected by the actiondetection means.

[0024] With this action controlling system for the robot apparatus, therobot apparatus detects the information contained in the action outputby the counterpart robot apparatus by action detection means to outputthe action corresponding to the information as detected by the actiondetection means by the action outputting means. This action controllingsystem for the robot apparatus permits the robot apparatus to actresponsive to the action of the robot apparatus.

[0025] For accomplishing the above object, an action controlling methodfor the robot apparatus according to the present invention-includes anaction outputting step of one of the robot apparatus outputting a presetaction when one of the robot apparatus is at a preset state, and areactive action outputting step of the other robot apparatus outputtingan action corresponding to the preset action output by the one robotapparatus. This action controlling method for the robot apparatuspermits the robot apparatus to act responsive to the action of the robotapparatus.

[0026] Other objects, features and advantages of the present inventionwill become more apparent from reading the embodiments of the presentinvention as shown in the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0027]FIG. 1 illustrates a conventional robot apparatus designed forchanging the pronunciation depending on the emotion.

[0028]FIG. 2 illustrates a conventional robot apparatus which has itsaction controlled by, e.g., a scale command.

[0029]FIG. 3 is a perspective view showing the appearance of a robotapparatus embodying the present invention.

[0030]FIG. 4 is a block diagram showing a circuit structure of the robotapparatus.

[0031]FIG. 5 is a block diagram showing a software structure of therobot apparatus.

[0032]FIG. 6 is a block diagram showing the structure of a middle warelayer in the software structure of the robot apparatus.

[0033]FIG. 7 is a block diagram showing the structure of an applicationlayer in the software structure of the robot apparatus.

[0034]FIG. 8 is a block diagram showing the structure of a behavioralmodel library of the application layer.

[0035]FIG. 9 illustrates a finite probability automaton as theinformation for behavior decision for the robot apparatus.

[0036]FIG. 10 shows a status transition table provided for each node ofthe finite probability automaton.

[0037]FIG. 11 illustrates a robot apparatus which enables the dialogwith other robot apparatus having a different dialog function.

[0038]FIG. 12 shows an illustrative structure of a database for soundscale commands provided on the robot apparatus.

[0039]FIG. 13 illustrates a robot apparatus capable of having a dialogby the same dialog function.

[0040]FIG. 14 shows dialog exchange in case both a main robot apparatusand a recipient robot apparatus are happy.

[0041]FIG. 15 shows dialog exchange in case the master robot apparatusis angry and the recipient robot apparatus is happy.

[0042]FIG. 16 shows dialog exchange in case the master robot apparatusissues a command and the recipient robot apparatus operates responsiveto the command.

BEST MODE FOR CARRYING OUT THE INVENTION

[0043] Referring to the drawings, a preferred embodiment of the presentinvention will be explained in detail. The present embodiment isdirected to an autonomous robot apparatus which acts autonomouslyresponsive to the external information, ambient environment (orextraneous factors) or to internal states (or internal factors).

[0044] In the present embodiment, the structure of the robot apparatusis first explained and subsequently applying part of the presentinvention in the robot apparatus is explained in detail.

(1) Structure of Robot Apparatus of the Present Embodiment

[0045] As shown in FIG. 3, the robot apparatus is a so-called pet robot,simulating an animal, such as a ‘dog’, and is constructed by leg units3A, 3B, 3C and 3D, connected on the front and rear sides on the left andright sides of a trunk unit 2, and by a head unit 4 and a tail unit 5,connected to the front and rear ends of the trunk unit 2, respectively.

[0046] Referring to FIG. 4, the trunk unit 2 includes a controller unit16, comprised of an interconnection over an internal bus 15 of a CPU(central processing unit) 10, a DRAM (dynamic random access memory) 11,a flash ROM (read-only memory) 12, a PC (personal computer) cardinterface circuit 13 and a signal processing circuit 14, and a battery17 as a power supply for the robot apparatus 1. In the trunk unit 2 arealso housed an angular velocity sensor 18 and an acceleration sensor 19for detecting the posture and the acceleration of movement of the robotapparatus 1.

[0047] On the head unit 4, there are mounted, in position, a CCD (chargecoupled device) camera 20 for imaging an outside state, a touch sensor21, for detecting the pressure resulting from a physical action, such as‘stroking’ or ‘patting’ from the user, a distance sensor 22 formeasuring the distance to an object positioned ahead, a microphone 23for collecting the external sound, a loudspeaker 24 for outputting thesound, like whining, and LEDs (light emitting diodes)equivalent to the‘eyes’ of the robot apparatus 1.

[0048] The joint portions of the leg units 3A to 3D, connecting portionsof the leg units 3A to 3D and the trunk unit 2, connecting portions ofthe head unit 4 and the trunk unit 2 and the connecting portion of atail SA of the tail unit 5 are provided with a number of actuators 25 ₁to 25 _(n) and potentiometers 26 ₁ to 26 _(n) corresponding to thenumber of the degrees of freedom. For example, the actuators 25 ₁ to 25_(n) include servo motors. The leg units 3A to 3D are controlled by thedriving of the servo motors to transfer to a targeted posture ormovement.

[0049] The sensors, such as the angular velocity sensor 18, accelerationsensor 19, touch sensor 21, floor contact sensors 23R/L, posture sensor24, distance sensor 25, microphone 26, distance sensor 22, microphone23, loudspeaker 24 and the potentiometers 25 ₁ to 25 _(n) are connectedvia associated hubs 27 ₁ to 27 _(n) to the signal processing circuit 14of the controller 16, while the CCD camera 20 and the battery 17 areconnected directly to the signal processing circuit 14.

[0050] The signal processing circuit 14 sequentially captures sensordata, picture data or speech data, furnished from the above-mentionedrespective sensors, to cause the data to be sequentially stored overinternal bus 15 in preset locations in the DRAM 11. In addition, thesignal processing circuit 14 sequentially captures residual batterycapacity data indicating the residual battery capacity supplied from thebattery 17 to store the data thus captured in preset locations in theDRAM 11.

[0051] The respective sensor data, picture data, speech data and theresidual battery capacity data, thus stored in the DRAM 11, aresubsequently utilized when the CPU 10 performs actional control of therobot apparatus 1.

[0052] In actuality, in an initial stage of power up of the robotapparatus 1, the CPU 10 reads out a memory card 28 loaded in a PC cardslot, not shown, of the trunk unit 2, or a control program stored in theflash ROM 12, either directly or through a PC card interface circuit 13,for storage in the DRAM 11.

[0053] The CPU 10 then checks its own status and surrounding statuses,and the possible presence of commands or actions from the user, based onthe sensor data, picture data, speech data or residual battery capacitydata, sequentially stored from the signal processing circuit 14 to theDRAM 11.

[0054] The CPU 10 also determines the next ensuing actions, based on theverified results and on the control program stored in the DRAM 11, whiledriving the actuators 25 ₁ to 25 _(n), as necessary, based on the sodetermined results, to produce behaviors, such as swinging the head unit4 in the up-and-down direction or in the left-and-right direction, ormoving the leg units 3A to 3D for walking or jumping.

[0055] The CPU 10 generates speech data as necessary and sends the sogenerated data through the signal processing circuit 14 as speechsignals to the loudspeaker 24 to output the speech derived from thespeech signals to outside or turns on/off or flicker the LEDs.

[0056] In this manner, the present robot apparatus 1 is able to behaveautonomously responsive to its own status and surrounding statuses, orto commands or actions from the user.

(2) Software Structure of Control Program

[0057]FIG. 5 shows the software structure of the above-mentioned controlprogram in the robot apparatus 1. In FIG. 5, a device driver layer 30 ispositioned in the lowermost layer of the control program, and is formedas a device driver set 31 made up by plural device drivers. Each devicedriver is an object allowed to directly access the hardware used in aroutine computer, such as an CCD camera 20 (FIG. 4) or a timer, andperforms processing responsive to interruption from an associatedhardware.

[0058] A robotics server object 32 is made up by a virtual robot 33, apower manager 34, comprised of a set of software items responsible forswitching between power sources, a device driver manager 35, comprisedof a set of software items, supervising various other device drivers,and a designed robot 36, comprised of a set of software itemssupervising the mechanism of the robot apparatus 1. The virtual robot33, located in the lowermost layer of the device driver layer 30, iscomprised of a set of software items furnishing an interface foraccessing the hardware items, including the above-mentioned varioussensors and actuators 25 ₁ to 25 _(n).

[0059] A manager object 37 is made up by an object manager 38 and aservice manager 39. The object manager 38 is a set of software itemssupervising the booting and the end of respective software itemsincluded in the robotics server object 32, a middle ware layer 40 and anapplication layer 41, while the service manager 39 is a set of softwareitems supervising the connection to respective objects based on theinformation on the connection among respective objects stated in aconnection file stored in a memory card 28 (FIG. 4).

[0060] The middle ware layer 40 is positioned as an upper layer of therobotics server object 32, and is made up by a set of software itemsproviding basic functions of the robot apparatus 1, such as pictureprocessing or speech processing. The application layer 41 is located asan upper layer of the middle ware layer 40, and is a set of softwareitems for deciding on the behavior of the robot apparatus 1 based on theresults of the processing by the software items making up the middleware layer 40.

[0061]FIG. 6 and FIG. 7 shows specified software structures of themiddle ware layer 40 and the application layer 41.

[0062] Referring to FIG. 6, the middle ware layer 40 is made up by arecognition system 60, having signal processing modules 50 to 58 fornoise-, temperature- or lightness detection, sound scale recognition,distance- or posture detection, for a touch sensor, for motion detectionand for color recognition, and an input semantics converter module 68,and by an outputting system 69, having an output semantics convertermodule 68 and signal processing modules 61 to 67 for posture management,tracking, motion reproduction, walking, restoration from the falldownstate, LED lighting and for sound reproduction.

[0063] The signal processing modules 50 to 58 of the recognition system60 captures relevant data from the sensor data, picture data and thespeech data, read out by the virtual robot 33 of the robotics serverobject 32 from the DRAM 11 (FIG. 4) to process the data and routes theprocessed results to the input semantics converter module 59. It isnoted that the virtual robot 33 is constructed as a component forexchanging or converting signals in accordance with a presetcommunication protocol.

[0064] The input semantics converter module 59 recognizes the ownstatus, the surrounding status, user's commands or actions, such as‘annoying’, ‘sultry’, ‘light’, ‘a ball has been detected’, ‘falldown isdetected’, ‘stroked’, ‘patted’, ‘do-mi-so scale has been heard’, ‘amoving object has been detected’, or ‘an obstacle has been detected’ tooutput the results of recognition to the application layer 41 (FIG. 5).

[0065] The application layer 41 is made up by five modules, namely abehavioral model library 70, a behavioral switching module 71, alearning module 72, a emotional model 73 and an instinct model 74, asshown in FIG. 7.

[0066] In the behavioral model library 70 there are provided respectiveindependent behavioral models 70 ₁ to 70 _(n) in association with pluralpre-selected condition items, such as ‘residual battery capacity issmall’, ‘restoration from the faldown state’, ‘an obstacle is to beevaded’, ‘the emotion is to be expressed’ or ‘a ball has been detected’,as shown in FIG. 8.

[0067] When the results of recognition are provided from the inputsemantics converter module 59 or a preset time has elapsed as from thetime the last results of recognition were provided, the behavioralmodels 70 ₁ to 70 _(n) decide on the next behaviors, as they refer toparameter values of the emotion as held by the emotional model 73 and toparameter values of the corresponding desires as held by the instinctmodel 74, to send the results of decision to the behavioral switchingmodule 71.

[0068] In the present embodiment, the behavioral models 70 ₁ to 70 _(n)use an algorithm, termed finite probability automaton, as a technique ofdeciding on the next behavior. This algorithm probabilisticallydetermines from which one of the nodes (states) NODE₀ to NODE_(n) towhich one of these nodes NODE₀ to NODE_(n) transition is to be made,based on the values of the transition probability P₁ to P_(n) as set forthe arcs ARC₁ to ARC_(n) interconnecting the respective nodes NODE₀ toNODE_(n).

[0069] Specifically, each of the behavioral models 70 ₁ to 70 _(n)includes a status transition table 80, shown in FIG. 10, for each of thenodes NODE₀ to NODE_(n), forming the own behavioral models 70 ₁ to 70_(n), in association with these nodes NODE₀ to NODE_(n).

[0070] In the status transition table 80, input events (results ofrecognition), as the conditions for transition in the nodes NODE₀ toNODE_(n), are listed in the column of the ‘input event name’, in thepriority order, and further conditions for the transition condition arestated in associated rows of the columns ‘data name’ and ‘data range’.

[0071] Thus, in the node NODE₁₀₀, shown in the status transition table80 of FIG. 10, given the results of recognition ‘ball has been detected’(BALL), the ball size (SIZE) being ‘from 0 to 1000’, as given along withthe results of recognition, represents a condition for transition toanother node. Similarly, given the results of recognition ‘an obstaclehas been detected’ (OBSTACLE), the distance (DISTANCE) to the obstaclebeing in a range ‘from 0 to 100’, as given along with the results ofrecognition, represents a condition for transition to another node.

[0072] Also, in the present node NODE₁₀₀, if no results of recognitionare input, but any one of the parameter values ‘joy’ (JOY), surprise(SURPRISE) or ‘sadness’ (SADNESS) among the parameter values of therespective emotion and desires, as held in the emotional model 73, amongthe parameter values periodically referenced by the behavioral models 70₁ to 70 _(n), is in a range between ‘50 and 100’, transition may be madeto another node.

[0073] Moreover, in the status transition table 80, the node names towhich transition can be made from the node NODE₀ to NODE_(n) are shownin the row ‘nodes of destination of transition’ in the column‘probability of transition to other nodes’. Additionally, theprobability of the transition to other nodes NODE₀ to NODE_(n), enabledwhen all conditions stated in the columns ‘input event name’, ‘dataname’ and ‘data range’ are met, is entered in corresponding locations inthe column ‘probability of transition to other nodes’. The behaviors tobe output on the occasion of transition to the nodes NODE₀ to NODE_(n)are indicated in the row ‘output behavior’ in the column ‘probability oftransition to other nodes’. Meanwhile, the sum of the probability valuesof each row in the column ‘probability of transition to other nodes’ is100%.

[0074] Thus, in the node NODE₁₀₀ represented by the status transitiontable 80 of FIG. 10, given the results of recognition that ‘the ball hasbeen detected’ and that the size (SIZE) of the ball is in a range from‘0 to 1000’, transition to the ‘node NODE₁₂₀ (node 120)’ can be madewith the probability of 30%, and the behavior ‘ACTION 1’ is then output.

[0075] In each of the behavioral models 70 ₁ to 70 _(n) a plural numberof the sets of the nodes NODE₀ to NODE_(n), each stated as this statustransition table 80, are concatenated together, such that, given theresults of recognition from the input semantics converter module 59, thenext behavior is probabilistically determined by exploiting the statustransition tables of the NODE₀ to NODE_(n) and the results of thedecision are output to the behavioral switching module 71.

[0076] The behavioral switching module 71, shown in FIG. 7, sends to theoutput semantics converter module 68 of the middle ware layer 40 acommand to select the behavior output from one of the behavioral models70 ₁ to 70 _(n), having a preset high priority order, among thebehaviors output from the respective behavioral models 70 ₁ to 70 _(n)of the behavioral model library 70, and to execute the behavior. Thiscommand is referred to below as a behavioral command. In the presentembodiment, the order of priority of a given one of the behavioralmodels 70 ₁ to 70 _(n) shown in FIG. 8 is the higher the lower the rankof the behavioral model in question in FIG. 8.

[0077] The behavioral switching module 71 notifies the learning module72, emotional model 73 and the instinct model 74 of the effect of thetermination of the behavior, based on the behavior completioninformation afforded from the output semantics converter module 68 afterthe end of the behavior.

[0078] The learning module 72 is fed with the results of recognition ofthe instructions received as an action from a user, such as ‘patting’ or‘stroking’, from among the results of recognition provided from theinput semantics converter module 59.

[0079] The learning module 72 changes the probability of transition ofthe behavioral models 70 ₁ to 70 _(n) in the behavioral model library70, based on the results of recognition and on the notification from thebehavioral switching module 71, such that, when the action is ‘patting’(‘scolding’) or‘stroking’ (‘praising’), the probability of occurrence ofthe behavior in question will be increased or decreased, respectively.

[0080] On the other hand, the emotional model 73 is holding parametersrepresenting the intensity of each of the six emotion types, namely joy(JOY), sadness (SADNESS), anger (ANGER), surprise (SURPRISE), disgust(DISGUST) and fear (FEAR). The emotional model 73 periodically updatesthe parameter values of these emotion types, based on the particularresults of recognition provided by the input semantics converter module59, such as ‘patted’ or ‘stroked’, time elapsed and on the notificationfrom the behavioral switching module 71.

[0081] Specifically, the emotional model 73 calculates a parameter valueE[t+1] of the current emotion type for the next period in accordancewith the following equation (1):

E[t+1]=E[t]+ke×ΔE[t]  (1)

[0082] where ΔE[t] in the amount of variation of the emotion type ascalculated by a preset equation based on, for example, the results ofrecognition provided by the input semantics converter module 59, thebehavior of the robot apparatus 1 at the pertinent time or on the timeelapsed as from the previous updating event, E[t] is the currentparameter value of the emotional type and ke is a coefficientrepresenting the sensitivity of the emotion type. The emotional model 73substitutes the so calculated value for the current parameter value E[t]of the emotion type to update the parameter value of the emotion type.In similar manner, the emotional model 73 updates the parameter valuesof the totality of the emotion types.

[0083] Which effect the respective results of recognition and thenotification from the output semantics converter module 68 will have onthe variation of the parameter values of the respective emotion typesΔE[t] is predetermined, such that the results of recognition ‘patted’significantly affects the amount of variation ΔE[t] of the parametervalue of the emotion type ‘anger’, while the results of recognition‘patted’ significantly affects the amount of variation ΔE[t] of theparameter value of the emotion type ‘joy’.

[0084] The notification from the output semantics converter module 68 isthe so-called behavior feedback information (behavior end information)and the information concerning the results of occurrence of thebehavior. The emotional model 73 also changes the emotion based on thisinformation. For example, the level of the emotion of anger may belowered by the act of ‘barking’. Meanwhile, the notification from theoutput semantics converter module 68 is also input to the learningmodule 72, which then changes the corresponding transition probabilityof the behavioral models 70 ₁ to 70 _(n) based on this notification.

[0085] Meanwhile, the feedback of the results of the behavior may bemade by an output of the behavioral switching module 71 (behaviorseasoned with the emotion).

[0086] On the other hand, the instinct model 74 holds the parameters,representing the intensity of four reciprocally independent desires,namely ‘desire for exercise’ ‘desire for affection’, ‘appetite’ and‘curiosity’. The instinct model 74 periodically updates the parametervalues of these desires, based on the results of recognition providedfrom the input semantics converter module 59, time elapsed and on thenotification from the behavioral switching module 71.

[0087] Specifically, as concerns the ‘desire for exercise’, ‘desire foraffection’ and ‘curiosity’, the instinct model 74 calculates, at apreset period, the parameter value I [k+1] of these desires at the nextperiod, using the following equation (2):

I[k+1]=J[k]+ki×ΔI[k]  (2)

[0088] where ΔI[k] is the amount of variation of the desire in questionat a pertinent time as calculated by a preset equation based on theresults of recognition, time elapsed and the notification of the outputsemantics converter module 68, I[k] is the current parameter value ofthe desire and ki is the coefficient representing the sensitivity of thedesire in question, and substitutes the calculated results for thecurrent parameter value I[k] to update the parameter value of thedesire. The instinct model 74 updates the parameter values of therespective desires except the ‘appetite’.

[0089] The effect of the results of recognition and the notificationfrom the output semantics converter module 68 on the amount of variationΔI[k] of the parameter values of the respective desires ispredetermined, such that, for example, the notification from the outputsemantics converter module 68 significantly affects the amount ofvariation ΔI[k] of the parameter values of ‘fatigue’.

[0090] In the present embodiment, the parameters of the respectiveemotion types and the respective desires (instincts) are varied in arange from 0 to 100, while the values of the coefficients ke and ki arealso set individually for the respective emotion types and forrespective desires.

[0091] The output semantics converter module 68 of the middle ware layer40 sends abstract behavioral commands, such as ‘go ahead’, ‘joy’, ‘cry’,or ‘tracking (track a ball)’, provided by the behavioral switchingmodule 71 of the application layer 41, as described above, to the signalprocessing modules 61 to 67 of the output system 69, as shown in FIG. 6.

[0092] Given a command for a behavior, the signal processing modules 61to 67 generates servo command values to be supplied to the associatedactuators 25 ₁ to 25 _(n) (FIG. 4) to execute the behavior, speech dataof the sound to be output from the loudspeaker 24 (FIG. 4) and/ordriving data to be supplied to the LED of the ‘eye’, based on thebehavioral command, and send these data through the virtual robot 33 ofthe robotics server object 32 and the signal processing circuit 14 (FIG.4) in this order to the associated actuators 25 ₁ to 25 _(n),loudspeaker 24 or to the LED.

[0093] Thus, the robot apparatus 1 is able to perform an autonomousbehavior, based on the control program, responsive to its own internalstate, surrounding state (exterior state) or to the command or actionfrom the user.

[0094] In the following explanation, the structure of the robotapparatus is first explained, and the portion thereof pertinent to thepresent invention is then explained.

(3) Structure Realized by the Application of the Present Invention

[0095] By the application of the present invention, the robot apparatus,having the above structure, is able to operate responsive to the actionof another robot apparatus. This enables the plural robot apparatus tohave dialog with one another. Moreover, the robot apparatus may have adialog with another robot apparatus having a different dialog function.First, the robot apparatus of the present invention, according to thepresent invention, capable of having a dialog with the robot apparatusof the different dialog function, is first explained, and the robotapparatus according to the present invention, capable of having a dialogbased on the same dialog function, is subsequently explained.

(3-1) Robot Apparatus Having a Dialog with a Robot Apparatus Having aDifferent Dialog Function

[0096] The robot apparatus according to the present invention is able tohave a dialog with another robot apparatus having a different dialogfunction.

[0097] The robot apparatus having a different dialog function may beexemplified by a robot apparatus of a different machine type, a robotapparatus furnished from a different manufacturer and a robot apparatusproduced by the same manufacturer but different in type number or model.In the present embodiment, explanation is made under an assumption thata dialog is to be had with another robot apparatus which has its dialogfunction controlled by the sound scale command. In this case, the otherrobot apparatus, as a counterpart of the dialog, utters, e.g.,‘pi-ro-lin’, in its standby state, as a preset action, on recognition ofthe scale command composed of scale sounds.

[0098] By the robot apparatus of the present embodiment of the presentinvention having such sound scale command outputting function, andreacting, such as by a synchronous action, to the action of anotherrobot apparatus, having its action controlled by the sound scalecommand, the user is in a position of appreciating the actions, producedas a result of reactions between the robot apparatus, as the dialogbeing had by the robot apparatus.

[0099] The robot apparatus of the present embodiment realizes the dialogwith the other robot apparatus in a manner which will be clarified inthe following explanation. In this explanation, the robot apparatus ofthe present embodiment is termed a main robot apparatus and the robotapparatus having the dialog function different from that of the mainrobot apparatus is termed a slave robot apparatus.

[0100] The main robot apparatus includes action detection means fordetecting the information contained in the actions output by the slaverobot apparatus as the other robot apparatus, and action outputtingmeans for outputting an action corresponding to the information asdetected by the action detection means. Specifically, the main robotapparatus includes a CPU 10, as described above. Specifically, the CPU10 implements the above means as an action detecting function ofdetecting the information contained in the action output by the slaverobot apparatus and an action outputting function of outputting theaction corresponding to the information as detected by the actiondetecting function. That is, the CPU 10 implements the above functionsby a variety of signal processing modules.

[0101] The main robot apparatus also includes communication means forexchanging the information with the slave robot apparatus. Theinformation here is that by the scale sound. Thus, the communicationmeans realizes the information transmission/reception by a loudspeaker24. The information is also the meaning information indicating themeaning of an action. For example, the information may be theinformation for the meaning of an action itself or the emotion containedin the action.

[0102] The main robot apparatus, having the above-described structure,realizes the dialog function with the slave robot apparatus by asequence of processing actions such as a so-called dialog mode.Specifically, the main robot apparatus realizes the dialog function byone of the behavior models shown in FIG. 8, for example, a dialogbehavior model.

[0103] The main robot apparatus, having such function, recognizes the‘emotion expressing sound’, such as ‘pi-ro-lin’, which is the soundsignal issued from the slave robot apparatus in an idling state ST1, asshown in FIG. 11. By this, the main robot apparatus detects the presenceof the slave robot apparatus and is thereby apprized of the slave robotapparatus being at a nearby position. This recognition indicates thatthe main robot apparatus is at a preset condition and is in a presetstate ready for commencing or firing the dialog function (for example,the dialog behavior model).

[0104] The idling state ST1 is such a state in which the main robotapparatus is acting autonomously, that is a state on which the robotapparatus is acting autonomously depending on the external environmentor responsive to its own emotional states.

[0105] On recognition of the emotion expressing sound, uttered by theslave robot apparatus, the main robot apparatus causes the slave robotapparatus to transfer to a ‘sit’ posture. It should be noted that themain robot apparatus does not necessarily have to cause the slave robotapparatus to transfer to the ‘sit’ posture but may cause the slave robotapparatus to transfer to other postures or to continue its autonomousbehavior.

[0106] The main robot apparatus then causes the transfer to the commandproducing state ST2 to produce a sound scale command comprised of scalesounds. For example, the main robot apparatus generates the scalecommand as random.

[0107] Specifically, the main robot apparatus includes a databasecomprised of various sound scale commands. It should be noted that thevarious sound scale commands are associated with respective differentaction contents to be executed by the slave robot apparatus, with thescale sounds corresponding to different combinations of the sound names.The sound scale commands, forming the database owned by the main robotapparatus, are the sound scale commands owned by a so-called soundcommander in order to control, e.g., the slave robot apparatus.

[0108] For example, the main robot apparatus includes a database made upby patterns of scale sounds of the sound scale command and by thecommand contents, as shown in FIG. 12. The command contents are actioncontents, such as uttering actions, performed by the slave robotapparatus on recognition of the corresponding sound scale patterns.Meanwhile, the commands implemented by the sound scale commands are notlimited to the speech uttering actions, but may also be directed to, forexample, posture changing actions.

[0109] The sound scale commands, owned in this manner by the main robotapparatus, may be registered during manufacture, but may also besubsequently registered anew or updated. The main robot apparatusselects the sound scale commands, owned as, e.g., a database, at random,and outputs the so selected sound scale command.

[0110] The main robot apparatus may also output the sound scale commandsin association with actions. Usually, the sound scale commands areretained to be signal patterns for controlling the robot apparatus, andare handled as being meaningful for the robot apparatus. However, thesecommands are not meaningful, insofar as the human being is concerned.Thus, by causing the main robot apparatus to output the actionsassociated with the sound scale commands, the user is able to comprehendthe meaning of the sound scale commands.

[0111] On outputting the sound scale command, the main robot apparatusenters into a state ST3 of waiting for reactions of a slave robotapparatus. The main robot apparatus maintains its reaction awaitingstate ST3 for a preset time.

[0112] On recognition of the sound scale command, issued by the mainrobot apparatus, the slave robot apparatus outputs a emotion expressingsound, depending on the sound scale command. However, it is indefinitewhether or not the slave robot apparatus necessarily responds to thesound scale command. The reason is that the slave robot apparatus mayhave gone away to some distant place by its own autonomous behavior orcannot detect the sound scale command due to, for example, ambientnoise. In order to cope with this, the main robot apparatus provides fora variety of actions, responsive to reactions of the slave robotapparatus.

[0113] When the slave robot apparatus has output a emotion expressingsound, responsive to a sound scale command issued by the main robotapparatus, the latter recognizes this emotion expressing sound andoutputs the same emotion expressing sound. For example, the main robotapparatus reads out and outputs the emotion expressing sound of theslave robot apparatus it holds as a database from the outset.

[0114] The user is able to appreciate the action performed between themain robot apparatus and the slave robot apparatus as the dialog betweenthese robot apparatus.

[0115] On recognition of the emotion expressing sound (sound scalereaction sound), the main robot apparatus increments the variableNumResponse by one, while setting a variable NumTimup to zero. It shouldbe noted that the variable NumResponse is a variable indicating thenumber of times of the reaction of the slave robot apparatus. Thisvariable NumResponse is afforded with zero as an initial value whenentering into the processing or mode of the dialog function. On theother hand, the variable NumTimup is a value indicating the number oftimes of failure of the slave robot apparatus to react to the soundscale command issued by the main robot apparatus. This value is referredto below as the number of times of non-reaction. For example, thefailure of the slave robot apparatus to react to the command is decidedon by the absence of the reaction from the slave robot apparatus for apreset time duration. This variable NumTimup is also afforded with zeroas an initial value when entering into the processing or mode of thedialog function.

[0116] The emotion recognition by the slave robot apparatus is byproviding a table showing the sound of the sound scale reaction and thecorresponding emotion and by referring the detected sound scale reactionsound to the table. It is also possible not to provide such tables, inwhich case the emotion is to be recognized based on analyses of thepitch, power etc. of the detected sound of the sound scale reaction.

[0117] When the slave robot apparatus 1 has made reactions, the mainrobot apparatus performs preset actions as the next following reactionaction.

[0118] The main robot apparatus performs the actions depending on thevalue of the above-mentioned NumResponse. The main robot apparatus usesthis variable NumResponse as a variable for deciding on an action.Specifically, the main robot apparatus has various actions as action(1), action (2), action (3), . . . , and decides on one of them by theaction (NumResponse) as an action on the slave robot apparatus.

[0119] The variable NumResponse indicates the number of times ofreaction of the slave robot apparatus as recognized by the main robotapparatus. Thus, the larger this number, the larger is the number oftimes of reaction of the slave robot apparatus. By and large, a reactionto a person's action expressing his or her happy state, if any, giveshim or her the happy emotion, as in case of chat or conversation. Inthis consideration, the actions of the action (1), action (2), action(3), . . . are defined so that the action indicating ‘happiness’ will beincreasingly garish in this order.

[0120] In the state of reaction action ST4, the main robot apparatusselects the action (NumResponse), from the action as defined above,depending on the current variable NumResponse, to perform the action.That is, if the variable NumResponse is 1 to 5, the main robot apparatuscauses the corresponding action (NumResponse) to be performed totransfer to the command issuing state ST2 again to select the soundscale command at random to output the so selected sound scale command.In the reaction awaiting state ST3, the main robot apparatus waits forthe reaction from the slave robot apparatus. If the slave robotapparatus makes a reaction, the main robot apparatus causes a presetaction to be performed again, responsive to the reactive action of theslave robot apparatus, to increment the variable NumResponse by 1, as anexample. The main robot apparatus then performs an action indicatinghappiness which has become more garish responsive to the variableNumResponse incremented by 1.

[0121] When the variable NumResponse exceeds 5, the main robot apparatusperforms the action of ‘banzai’ (‘hurrah!’), as an action (6), to thenrevert to the idling state ST1. That is, the main robot apparatuscompletes the processing or mode for having dialog with the slave robotapparatus to then revert to a mode for autonomous behavior.

[0122] Thus, the main robot apparatus is responsive to the reaction fromthe slave robot apparatus to perform variable actions depending on thenumber of times of the reactions. The user is able to appreciate theactions exchanged between the main robot apparatus and the slave robotapparatus as the dialog between the robot apparatus.

[0123] It should be noted that the action of the main robot apparatusindicating its ‘happiness’ is rendered increasingly garish with anincreasing number of times of reactions from the slaver robot apparatus.This may be grasped as what may be called the action modulation.

[0124] If the reaction from the slave robot apparatus cannot berecognized within a preset time duration in the reaction awaiting stateST3, the main robot apparatus performs an action indicating ‘sadness’.The robot apparatus also increments the variable NumTimUp, indicatingthe number of times of non-reaction, by 1. The case where there is noreaction from the slave robot apparatus may be such a case where theslave robot apparatus has gone away to some distant place by its ownautonomous behavior or may also be such a case where the slave robotapparatus cannot detect the sound scale command due to, for example,ambient noise.

[0125] Meanwhile, even when the slave robot apparatus fails to makereactions, it is expressing the action indicating ‘sadness’. It may besaid that, in such case, the main robot apparatus recognizes the failureof the slave robot apparatus by a reaction action.

[0126] Lacking the reactions from the slave robot apparatus, the mainrobot apparatus performs the aforementioned action and again selects asound scale command at random at the command issuing state ST2 to outputthe so selected sound scale command to wait again for the reaction fromthe slave robot apparatus in the reaction awaiting state St3. If, inthis reaction awaiting state St3, the main robot apparatus has beenunable to recognize the reactions from the slave robot apparatus for apreset number of times on end, that is if the number of times ofnon-reaction has reached a preset value, it performs an action ofincreased sadness. In the present instance, when the reaction from theslave robot apparatus cannot be recognized four times on end, that is ifthe variable NumTimUp, indicating the number of times of non-reaction,exceeds 3, the main robot apparatus performs the action indicating‘increased sadness’ to then revert to the idling state ST1.

[0127] Thus, lacking the reaction from the slave robot apparatus, themain robot apparatus performs variable actions, depending on the numberof times of non-reactions, whereby the user is able to appreciate theactions exchanged between the main and slave robot apparatus as theinteraction between the robot apparatus.

[0128] Meanwhile, in the present embodiment, the case in which theaction of ‘increased sadness’ is to be performed from the aforementionedreaction awaiting state ST3 is such a case where, in the reactionawaiting state ST3 to which the main robot apparatus transferred aplural number of times, the reaction from the slave robot apparatus hasnot been detected on end. It is because the variable NumTimUp indicatingthe number of times of non-reaction is initialized to zero when theslave robot apparatus has made response in the reaction awaiting stateST3. If, lacking a reaction from the slave robot apparatus to an actionfrom the main robot apparatus, the slave robot apparatus has reacted tosubsequent actions from the main robot apparatus, the main robotapparatus performs the action ‘happiness’ and initializes the variableNumTimUp, indicating the number of times of non-reaction, to zero.

[0129] The above-described processing is, of course, not limitative. Forexample, the number of times of non-reactions can be continuouslycounted without initializing the variable NumTimUp to zero responsive toreactions from the slave robot apparatus, that is, without dependency onwhether there has been or has not been made a reaction.

[0130] Thus, the main robot apparatus is able to perform variousactions, depending on the reacting actions of the slave robot apparatusto the actions from the main robot apparatus, inclusive of thenon-reacting actions, and on the number of times of non-reactions. Thisrenders it possible for the user to appreciate the actions performed asa result of reactions between the robot apparatus, as the dialog beingmade between the robot apparatus.

[0131] The main robot apparatus achieves this dialog by controlling anoriginal model or a robot apparatus produced by another manufacturer,the action of which is controlled by the external control command, suchas a sound scale command, in synchronism with its own actions. As aresult, the user is able to appreciate the reacting actions betweendifferent models of the robot apparatus of the same manufacturer orbetween robot apparatus of different manufacturers as the dialog betweenthe robot apparatus.

(3-2) Robot Apparatus Having a Dialog with the Same Dialog Function

[0132] The dialog which is made on the basis of the dialog functionbetween two robot apparatus, according to the present invention, ishereinafter explained. Each apparatus, explained here, has the samedialog function, which enables the two robot apparatus to have thedialog with each other.

[0133] Each robot apparatus includes action detection means fordetecting the information contained in the action output by thecounterpart robot apparatus, and action outputting means for outputtingthe actions corresponding to the information detected by the actiondetection means. Specifically, the main robot apparatus realizes this byits CPU as an action detecting function of detecting the informationcontained in the action output by the counterpart robot apparatus, andas an action outputting function of outputting the action correspondingto the information as detected by the action detection function. Thatis, the CPU 10 achieves these functions by a variety of signalprocessing modules.

[0134] Both robot apparatus are configured for outputting a presetaction when in a preset state. Thus, when in a preset state, one of therobot apparatus outputs a preset action by the action outputtingfunction, with the other robot apparatus outputting an actioncorresponding to the action output by the first-stated robot apparatus.

[0135] The two robot apparatus are of similar structure to each other toachieve the dialog with the counterpart robot apparatus. The dialogbetween the two robot apparatus is hereinafter explained with referenceto FIG. 13.

[0136] Both of the robot apparatus are in the idling states ST11, ST21,as shown in FIG. 13. When in a preset state, one of the robot apparatusperforms its action, whereby the dialog between the two robot apparatuscommences. The robot apparatus being in the preset state herein means,for example, the robot apparatus being of a certain value of theprobability or the level of the emotion, or the robot apparatusrecognizing the existence of the other robot apparatus. The level of theemotion is the parameter value of the above-mentioned emotion. The robotapparatus being in this preset state may be detected by monitoring at apreset sampling interval. If the preset state cannot be detected at thissampling timing, the main robot apparatus may deem such state as timeoutto revert to the idling state ST11. The robot apparatus which is in thepreset state and first performed the action for the dialog is termed a‘master, with the other robot apparatus being termed a ‘recipient’.

[0137] When in the preset state, each robot apparatus outputs a presetaction. Thus, depending on the timing of being in the preset state, the‘other robot apparatus’ may be a master, with the remaining robotapparatus being a recipient. That is, each of the robot apparatus maybecome a ‘master’ or a ‘recipient’ depending on circumstances.

[0138] The robot apparatus which has been the first to be in the presetstate to start or fire the dialog function to become a master utters‘Yahhoo!’ and the ‘scale sound (1)’, as an action ST12. It should benoted that the ‘Yahhoo!’ is the enunciation as the language, while the‘scale sound (1)’ is the scale sound, such as ‘pi-po-paa’, that is noother than a sound sequence. In the following explanation, scale sounds(2), (3) and (4) . . . , are given as examples of the scale sounds. Eachof these is of a different sound scale pattern. The scale sounds (2),(3) and (4) . . . , represent techniques established in the prior art asthe sound scale commands.

[0139] Meanwhile, the scale sounds are output in association with theenunciation such as ‘Yahhoo!’ in such a case where the recipient robotapparatus as a counterpart of the dialog cannot recognize the language‘Yahhoo!’ as the enunciation action as being meaningful. Thus, the scalesounds are not necessary if the robot apparatus is able to recognize theaction as being meaningful. That is, it is when the action cannot beunderstood by the counterpart apparatus as being meaningful, that iswhen the enunciation as the action is a long text, that the scale soundis output simultaneously with the action. In such case, the scale soundoperates as effective dialog means between the robot apparatus.

[0140] The recipient recognizes such enunciation of the master totransfer from an idling state ST21 to a reactive action ST22. That is,the recipient boots its dialog function, with the master's enunciationas a trigger.

[0141] The recipient enunciates ‘Yahhoo!’ and the scale sound (2)’ asthe reactive action ST22. The recipient recognizes the enunciation madeby the master by the ‘scale sound (1)’ of the ‘Yahhoo!’ and the ‘scalesound (1)’ as enunciated by the master.

[0142] The scale sound is the technique already established as the soundscale command in the control robot apparatus, as described above. By theapplication of the scale sound recognition system, thus established, therobot apparatus 1 accurately recognizes the command through a scalesound pattern of the scale sounds.

[0143] The master recognizes the enunciation of the ‘scale sound (2)’ ofthe recipient and enunciates ‘happy’ and ‘the sound (3)’ as its reactiveaction ST13. The reactive actions to the action of the counterpart sideare repeated as the contents of the reactive actions are changed.

[0144] That is, the recipient recognizes the enunciation of the ‘scalesound (3)’ of the master and utters ‘calm’ and ‘the scale sound (4)’ asthe reactive action ST23. Meanwhile, ‘calm’ is the enunciationindicating the contents of, for example, ‘be calm’. The masterrecognizes the enunciation of the ‘scale sound (4)’ and utters ‘happy!’and ‘the scale sound (5)’ as its reactive action ST14. Meanwhile,‘happy!’ is no more than the emphasized expression of ‘happy’. Therecipient recognizes the enunciation of the master of ‘the scale sound(5)’ to utter ‘happy’ and ‘the scale sound (6)’ as its reactive actionST24. The master then recognizes the enunciation of the ‘scale sound(6)’ of the recipient to utter ‘satisfied’ as the last action of thedialog action. In keeping therewith, the recipient similarly utters‘satisfied’ in synchronism with the master's enunciation.

[0145] In this manner, the two robot apparatus, operating as a masterand as a recipient, are able to react to the counterpart side's action,whereby the user is able to appreciate the action performed as a resultof reaction between the robot apparatus as the dialog between the robotapparatus.

[0146] By exploiting the recognition system of the scale sound,established as the technique, the counterpart side's action can berecognized, as described above. This, of course, is not limitative, suchthat one of the robot apparatus is also able to directly recognize themeaningful language uttered by the robot apparatus, such as ‘Yahhoo!’,to take a reactive action, whereby the user is able to grasp the dialogby the robot apparatus as being more realistic dialog.

[0147] Lacking the reaction from the counterpart robot apparatus withina preset time, this state may be deemed to be a timeout state to revertagain to idling.

[0148] In FIGS. 14 to 16, the reactive actions between the robotapparatus (master robot apparatus and the recipient robot apparatus),realized by the above-described dialog function, are indicatedchronologically.

[0149]FIG. 14 shows dialog exchange in case both the master and therecipient are happy, FIG. 15 shows that in case the master is angry andthe recipient is being scolded and FIG. 16 shows that in case the masterissues a command and the recipient operates in response thereto. Thesevariable dialog functions are implemented as the behavior model.

[0150] Specifically, when the master utters ‘Yahhoo!’, the recipientutters ‘Yahhoo!’ as a reactive action. Meanwhile, if the enunciation isfelt to be possibly understood with relative ease, the scale sound neednot be output. That is, if the enunciation such as ‘Yahhoo!’ can berecognized readily, the master utters only ‘Yahhoo!’, withoutcorrespondingly outputting the scale sound, with the counterpart siderobot apparatus then recognizing this ‘Yahhoo!’.

[0151] The master is responsive to the recipient's enunciation ‘Yahhoo!’to utter ‘Happy’ and the scale sound (sound scale command)’, while therecipient responsive to the enunciation ‘Happy’ and ‘the scale sound’ ofthe master utters ‘calm’ and ‘the scale sound’. The master responsive tothe recipient's enunciation ‘Happy’ and ‘the scale sound’ againrecognizes the enunciation ‘the scale sound’ utters ‘Happy’ and ‘thescale sound’. The recipient is responsive to the master's enunciation‘Happy!’ and the ‘scale sound’ to recognize the ‘scale sound’ to utter‘Happy’ and ‘the scale sound’. The master responsive to the enunciation‘Happy’ and ‘the scale sound’ of the recipient utters ‘comfort’ as thelast action of the dialog, whilst the recipient utters ‘comfort’, insynchronism therewith, as the last action of the dialog.

[0152]FIG. 15 shows an exchange in case the master is angry and therecipient is being scolded. In this case, when the master utters‘Yahhoo!’, the recipient utters ‘Yahhoo!’, as a reactive action. Themaster responsive to the recipient's enunciation ‘Yahhoo!’ utters‘anger’ and ‘the scale sound (scale sound command)’, while the recipientresponsive to the master's enunciation ‘anger’ and ‘the scale sound’recognizes the enunciation ‘scale sound’ to utter ‘calm’ and ‘the scalesound’. The master responsive to the recipient's enunciation ‘calm’ and‘scale sound’ recognizes the enunciation ‘scale sound’ to utter ‘angry’and ‘the scale sound’. The recipient responsive to the master'senunciation ‘anger’ and ‘the scale sound’ recognizes the ‘scale sound’to utter ‘sad’ and the ‘scale sound’. The master responsive to theenunciation ‘sad’ and the ‘scale sound’ utters ‘anger’ as the lastaction of the dialog. In synchronism therewith, the recipient utters‘sad’ as the ultimate action of the dialog.

[0153] As for exchange in case the master issues a command and therecipient operates responsive thereto, shown in FIG. 16, the masterfirst utters ‘Yahhoo!’, while the recipient is responsive thereto andutters ‘Yahhoo!’. The master raises his or her right hand (right forelimb), and utters ‘scale sound (sound scale command) responsive to therecipient's enunciation ‘Yahhoo!’, while the recipient is responsive tothe master's action of raising his or her right hand (right fore limb)and the enunciation ‘scale sound (scale sound command) to recognize theenunciation ‘sound scale’ to perform the action of rightward rotation ofhis or her body.

[0154] Although the recipient is able to apprize the master of the endof the action by the scale sound, the master may also perform timemanagement to detect the end of recipient side's action. For example,the master may hold the time of execution of the rightward rotation ofthe recipient's body, measure the time elapsed as from the time ofraising the right hand or uttering the scale sound and compare themeasured time with the time of execution of the rightward rotation itholds to grasp the end of the rightward trunk rotation of the recipientactually proceeding.

[0155] After detection of the end of the recipient's rightward rotation,the master performs the action of raising the left hand (left forelimb), while uttering the ‘scale sound (sound scale command). Therecipient is responsive to the master's action of raising the left hand(left fore limb), and to the ‘scale sound (sound scale command) torecognize the enunciation of the ‘scale sound’ to perform the action ofrotating its trunk leftwards. The master then measures the time elapsedsince raising the left hand or since the enunciation of the scale soundand compares it to the time of execution of the leftward rotation itholds to be apprized of the end of the leftward rotation the recipientis actually performing.

[0156] After detecting the end of the recipient's leftward rotation, themaster performs the action of raising both hands (both fore limbs),while uttering ‘scale sound (sound scale command)’. The recipient isresponsive to the master's action of raising both hands (both forelimbs) and the enunciation (sound scale command) to recognize theenunciation ‘sound scale sound’ to perform an action of moving forwards.

[0157] In this manner, the exchange proceeds in the case where themaster issues a command and the recipient operates responsive thereto.

[0158] As explained in the foregoing, with reference to Figs.14 to 16,the robot apparatus is able to realize various dialogs with thecounterpart side robot.

[0159] In the above-described embodiment, the communication between therobot apparatus takes place by the scale sound. This, however, is notlimitative, such that, for example, the information can be transmittedor received between robot apparatus by infrared rays.

[0160] Likewise, in the above-described embodiment, the dialog by theactions of the robot apparatus is made mainly based on the soundoutputting or speech uttering action. This again is not limitative suchthat the dialog may also be had by actuating movable parts, such aslimbs. In such case, the robot apparatus recognizes the counterpartside's action by image recognition means, such as CCD camera 20. If itis retained to be rather difficult to recognize the movement of therobot apparatus, the scale sound, for example, may be output in keepingwith the movement of the robot apparatus to permit the meaning of themovement action to be recognized by the scale sound to achieve thereliable dialog between robot apparatus.

[0161] If the robot apparatus autonomously generates the sentence toutter the speech, which may be complex and difficult to understand, thecounterpart robot apparatus is able to act reliably to comply with thecomplex text by simultaneously outputting the content information as thescale sound.

Industrial Applicability

[0162] According to the present invention, as described above, a robotapparatus is able to detect the information contained in an actionoutput by another robot apparatus by action detection means to causeaction outputting means to output an action corresponding to theinformation as detected by the action detection means, so that the robotapparatus is able to operate responsive to the action of other robotapparatus to improve the entertainment performance of the robotapparatus.

1. A mobile robot apparatus having an action part, comprising: actiondetection means for detecting the information contained in an actionoutput by another robot apparatus; action part control means forcontrolling said action part, based on the information detected by saidaction detection means; and means for measuring the number of times ofreactions of said other robot apparatus corresponding to the actionsoutput by said action part; said action part control means outputting anaction by said action part depending on the number of times of reactionsof said other robot apparatus.
 2. The robot apparatus according to claim1 wherein said information is the emotion information of said otherrobot apparatus.
 3. A robot apparatus comprising: action detection meansfor detecting the information contained in an action output by anotherrobot apparatus; and action outputting means for outputting an actioncorresponding to the information as detected by said action detectionmeans.
 4. The robot apparatus according to claim 3 wherein saidinformation is the information on the meaning of said action.
 5. Therobot apparatus according to claim 3 wherein the output action is anoutputting action of outputting a sound signal.
 6. The robot apparatusaccording to claim 3 wherein the robot apparatus changes the emotionbased on an external environment and/or the internal state and executesan autonomous behavior based on said emotion.
 7. The robot apparatusaccording to claim 3 wherein said other robot apparatus changes theemotion based on an external environment and/or on an internal state andexpresses the emotion in said action to output said action; and whereinsaid action detection means detects the emotion performed in said actionas said information.
 8. The robot apparatus according to claim 3 furthercomprising: communication means for transmitting/ receiving theinformation with said other robot apparatus.
 9. The robot apparatusaccording to claim 8 wherein said communication means sends theinformation on the meaning of said action in association with the actionoutput by said action outputting means.
 10. The robot apparatusaccording to claim 9 wherein the robot apparatus holds said meaninginformation detectable by said other robot apparatus as a database; saidcommunication means sending said meaning information in association withan action output by said action outputting means.
 11. The robotapparatus according to claim 8 wherein the robot apparatus receives bysaid communication means the meaning information of said actiontransmitted by said other robot apparatus, in association with an actionoutput by said other robot apparatus; said action detection meansdetecting said meaning information received by said communication meansas the information contained in said action.
 12. The robot apparatusaccording to claim 11 wherein the robot apparatus holds said meaninginformation transmitted by said other robot apparatus as a database;said action detection means selecting the contents of said meaning basedon said database from a signal received by said communication means todetect the so selected meaning information as the information containedin said action.
 13. The robot apparatus according to claim 8 whereinsaid communication means executes communication by the scale sound. 14.The robot apparatus according to claim 8 wherein said communicationmeans executes communication by infrared ray signals.
 15. The robotapparatus according to claim 3 wherein the action outputting meansoutputs a preset action when in a preset state.
 16. The robot apparatusaccording to claim 15 wherein the robot apparatus changes the emotionbased on an external environment and/or on the internal state; andwherein when the level of the emotion is at a preset level as saidpreset state, said action outputting means causes the emotion to beexpressed in its action to output the resulting action.
 17. The robotapparatus according to claim 15 wherein said preset action is outputwhen the presence of said other robot apparatus has been detected assaid preset state.
 18. The robot apparatus according to claim 3 whereinsaid action detection means detects the information contained in theaction output when the other robot apparatus is at a preset state. 19.The robot apparatus according to claim 3 wherein said action outputtingmeans responsive to said information detected by said action detectionmeans modulates a previous action, which has caused an action of saidother robot apparatus, in association with the information detected bysaid action detection means, to output the so modulated previous action.20. The robot apparatus according to claim 19 wherein said previousaction is an action of outputting the scale sound, and wherein saidaction outputting means modulates said scale sound to output the somodulated scale sound.
 21. A method for controlling the action of amobile robot apparatus having an action part, said method comprising: anaction detection step of detecting the information contained in anaction output by another robot apparatus; an action part controllingstep of controlling said action part, based on the information detectedby said action detections step; and a step of measuring the number oftimes of reactions of said other robot apparatus based on the actionsoutput by said action part; said action part controlling step outputtingan action by said action part depending on the number of times ofreactions of said other robot apparatus.
 22. The action controllingmethod according to claim 21 wherein said information is the emotioninformation of said other robot apparatus.
 23. A method for controllingthe action of a robot apparatus comprising: an action detection step ofdetecting the information contained in an action output by another robotapparatus and an action outputting step of causing one of the robotapparatus to output an action corresponding to the information asdetected by said action detection step.
 24. The action controllingmethod according to claim 23 wherein said information is the informationon the meaning of said action.
 25. The action controlling methodaccording to claim 23 wherein the output action is an action ofoutputting a sound signal.
 26. The action controlling method accordingto claim 23 further comprising: a transmission step of transmitting theinformation on the meaning of said action in association with an actionoutput at said action outputting step.
 27. The action controlling methodaccording to claim 23 further comprising: a receipt step of receivingthe information on the meaning of said action transmitted by said otherrobot apparatus in association with an action output by said other robotapparatus; said action detecting step detecting said meaning informationreceived at said receipt step as the information contained in saidaction.
 28. An action control system for a plurality of robot apparatusperforming an action corresponding to an action of a counterpart robotapparatus, said system comprising: a plurality of robot apparatusincluding action detection means for detecting the information containedin an action output by said counterpart robot apparatus; and actionoutputting means for outputting an action corresponding to saidinformation detected by said action detection means.
 29. The actioncontrol system for the robot apparatus according to claim 28 whereinsaid information is the information on the meaning of said action. 30.The action control system for the robot apparatus according to claim 28wherein said action outputting means outputs a preset action when theaction outputting means is in a preset state; and wherein one of therobot apparatus when in a preset state outputs a preset action by saidaction outputting means, the other robot apparatus outputting an actioncorresponding to said preset action output by said one robot apparatus.31. The action control system for the robot apparatus according to claim30 wherein said robot apparatus changes its emotion based on an externalenvironment and/or on an internal state; said action outputting meansexpressing the emotion on an action to output the resulting action whenthe preset level of the emotion as said preset state is reached.
 32. Theaction control system for the robot apparatus according to claim 30wherein said preset action is output when the presence of other robotapparatus is detected as said preset state.
 33. The action controlsystem for the robot apparatus according to claim 28 wherein said actionis the action of outputting the sound.
 34. The action control system forthe robot apparatus according to claim 28 wherein said action outputtingmeans modulates a previous action, which has caused an action of saidother robot apparatus, in association with said information detected bysaid action detection means, to output the so modulated previous action.35. The action control system for the robot apparatus according to claim28 wherein the robot apparatus includes communication means fortransmitting/receiving the information with a counterpart robotapparatus; and wherein one of the robot apparatus transmits the meaninginformation of said action, by said communication means, in associationwith an action output by said action outputting means; the other robotapparatus receiving said meaning information by said communication meansand detecting the meaning information by said action detection means asthe information contained in said action.
 36. The action control systemfor the robot apparatus according to claim 35 wherein said communicationmeans performs communication with the scale sound.
 37. The actioncontrol system for the robot apparatus according to claim 35 whereinsaid communication means performs communication with the infrared rays.38. A method for controlling the action of a robot apparatus inassociation with an action of a counterpart robot apparatus, said methodcomprising: an action outputting step of one of the robot apparatusoutputting a preset action when one of the robot apparatus is at apreset state; and a reactive action outputting step of the other robotapparatus outputting an action corresponding to said preset actionoutput by said one robot apparatus.
 39. The action control system forthe robot apparatus according to claim 38 wherein said information isthe information on the meaning of said action.
 40. The action controlsystem for the robot apparatus according to claim 38 wherein said onerobot apparatus is changing the emotion based on an external environmentand/or on the internal state; said one robot apparatus when at a presetlevel of the emotion which is said preset state causing ‘sad” emotion tobe expressed in an action to output the resulting action.